Solution of general linear systems of equations using block Krylov based iterative methods on distributed computing environments

We study the implementation of block Krylov based iterative methods on distributed computing environments for the solution of general linear systems of equations First we study potential implementation of the classical conjugate gradient CG method on parallel environments From the family of conjugate direction methods we study the Block Conjugate Gradient Block CG method which is based on the classical CG method The Block CG works on a block of s right hand side vectors instead of a single one as is the case of the Classical CG and we study the implementation of the Block CG on distributed environments The complexity of the Block CG method is higher than the complexity of the classical CG in terms of computations and memory requirements We show that the fact that an iteration of the Block CG requires more computations than the classical CG makes the Block CG more suitable for vector and parallel environments Additionally the increase in memory requirements is only a multiple of s which generally is much smaller than the size of the linear system being solved We present three models of distributed implementations of the Block CG and discuss the ad vantages and disadvantages from each of these model implementations The Classical CG and Block CG are suitable for the solution of symmetric and positive de nite systems of equations Furthermore both methods guarantee in exact arithmetic to nd a so lution to positive de nite systems in a nite number of iterations For non symmetric linear systems we study block row projection iterative methods for solving general linear systems of equations and we are particularly interested in showing that the rate of convergence of some row projection methods can be accelerated with the Block CG method We review two block projection methods the block Cimmino and the block Kaczmarz method Afterwards we study the implementation of an iterative procedure based on the block Cimmino method using the Block CG method to accelerate its rate of convergence on distributed com puting environments The complexity of the new iterative procedure is higher than the one of the Block CG method in terms of computations and memory requirements In this case the main advantage is the extension of the application of the CG based methods to general linear systems of equations We present a parallel implementation of the block Cimmino method with Block CG acceleration that performs well on distributed computing environments This last parallel implementation opens a study of potential scheduling strategies for distribut ing tasks to a set of computing elements We present a scheduler for heterogeneous computing environments which is currently implemented inside the block Cimmino based solver and can be reused inside parallel implementations of several other iterative methods Lastly we combine all of the above e orts for parallelizing iterative methods with preprocessing strategies to improve the numerical behavior of the block Cimmino based iterative solver